A common problem associated with capturing images with a handheld device (e.g., a camera phone) is that the images often become blurred (or distorted) as a result of the shaking of the handheld device due to hand jitter. Hand jitter is fundamental to human biology, and generally cannot be trained away, even for a professional photographer. The amount of blur in an image depends on many factors, including the amount of shaking and the length of the exposure time. Devices with low mass will tend to shake more, and devices with smaller pixels generally require longer exposure times. Current trends in handheld image capturing devices lead toward smaller and smaller units with smaller pixels, exacerbating the problem associated with blurred images. The consumer demand for higher pixel densities and optical zoom capabilities also increases the problem. For example, for a VGA resolution image capture system, a certain blur may be too small to be visible; but for a 3 MP (megapixel) image capturing system, the same blur will be easily apparent.
Various image stabilization techniques have been proposed to deal with distortion in an image. Image stabilization techniques can be generally categorized as optical image stabilization (OIS)—in which a lens or image sensor is mechanically moved in order to compensate for the shaking of a handheld device, digital image stabilization (DIS)—in which pure software is used to remove blur (or other distortion) from an image, or electronic image stabilization (EIS)—in which information from, e.g., a gyroscope, is used to augment a software algorithm to provide a stable image. Optical image stabilization is generally considered the most effective method for producing stable images as the technique mechanically prevents an image from becoming blurred through the use of actuators. However, conventional actuators are generally too large and expensive for adoption into smaller consumer devices such as camera phones. Convention digital image stabilization techniques for removing blur from an image using pure software requires substantial processing, and often results in images that are not useable.
With regard to electronic image stabilization, conventional systems typically read a series of images (or frames) from an image sensor, shift the images using gyroscope data, and combine the images to produce a single sharp image. However, such systems are limited by the read-out time of the image sensor and suffer non-linearities in pixel integration. For example, one conventional technique for combining a short exposure frame with a long exposure frame into a single frame is described in U.S. Patent Application Publication No. 2006/0017837, entitled “Enhancing Digital Photography”. According to this technique, the long exposure frame is used to provide data in which there are not many details, and the short exposure frame is used to provide fine details. However, this technique discards useful data from a long exposure frame that may be used to electronically stabilize an image. In addition, the long exposure frame may contain blur properties (e.g., non-linear blur properties) that are difficult to correct.